ABSTRACT
In response to the COVID-19 pandemic, the University of Leicester introduced a blended teaching model to continue delivery of their undergraduate Chemistry courses in 2020/21. The transition from in-person to blended provided a good opportunity to investigate student engagement in the blended environment, along with the attitudes of faculty members adapting to this mode of delivery. Data from 94 undergraduate students and 13 staff members was collected using surveys, focus groups, and interviews and analyzed using the community of inquiry framework. Analysis of the collected data found that, while some students felt unable to always engage and focus with the remote material, they were pleased with the University's response to the pandemic. Staff members commented on the challenges of gauging student engagement and understanding in synchronous contact sessions because students did not make use of cameras or microphones but praised the array of digital tools available that helped to facilitate some degree of student interaction. This study suggests there is scope for continuation and wider implementation of blended learning environments to provide additional contingency for further disruption to on-campus teaching and to provide new teaching opportunities, and it also presents recommendations as to how to reinforce the community of inquiry presences in blended learning.
ABSTRACT
BACKGROUND AND OBJECTIVES: Text-mining tool, Abstrackr, may potentially reduce the workload burden of title and abstract screening (Stage 1), using screening prioritization and truncation. This study aimed to evaluate the performance of Abstrackr's text-mining functions ('Abstrackr-assisted screening'; screening undertaken by a single-human screener and Abstrackr) vs. Single-human screening. METHODS: A systematic review of treatments for relapsed/refractory diffuse large B cell lymphoma (n = 7,723) was used. Citations, uploaded to Abstrackr, were screened by a human screener until a pre-specified maximum prediction score of 0.39540 was reached. Abstrackr's predictions were compared with the judgments of a second, human screener (who screened all citations in Covidence). The performance metrics were sensitivity, specificity, precision, false negative rate, proportion of relevant citations missed, workload savings, and time savings. RESULTS: Abstrackr reduced Stage 1 workload by 67% (5.4 days), when compared with Single-human screening. Sensitivity was high (91%). The false negative rate at Stage 1 was 9%; however, none of those citations were included following full-text screening. The high proportion of false positives (n = 2,001) resulted in low specificity (72%) and precision (15.5%). CONCLUSION: Abstrackr-assisted screening provided Stage 1 workload savings that did not come at the expense of omitting relevant citations. However, Abstrackr overestimated citation relevance, which may have negative workload implications at full-text screening.
Subject(s)
Data Mining , Workload , Humans , Data Mining/methods , Mass Screening , ResearchABSTRACT
IntroductionHuman screening of title and s in a systematic literature review (SLR) is labor intensive and time-consuming. In many instances, thousands of citations may be retrieved;the vast majority excluded upon screening. Text-mining semi-automates and accelerates screening by identifying patterns in relevant and irrelevant citations, as labelled by the screener. One such text-mining tool, Abstrackr, uses an algorithm within an active-learning framework to predict the likelihood of citations being relevant. The objective of this study was to assesses the performance of Abstrackr for title and screening in an SLR of treatments for relapsed/refractory diffuse large B-cell lymphoma.MethodsCitations identified from searches of electronic databases were imported to Abstrackr. An investigator-selected database of terms indicating relevance of title and to the research question were uploaded. These terms were partly informed by the SLR inclusion/exclusion criteria. Citations deemed most relevant by Abstrackr were screened first (screening prioritization). Screening was carried out until a maximum prediction score of 0.4 or less, based on previous experience in the literature, was reached. Remaining citations were deemed unlikely to be relevant and did not undergo screening (screening truncation). Separately, a single-human screener screened all citations using Covidence.ResultsA total of 7,723 citations and 154 initial terms were uploaded to Abstrackr. Of these citations, 2,572 (33 percent) were screened before a prediction score of 0.39 was reached. Compared to single-human screening (conducted on all citations), the workload saving associated with Abstrackr was 5 days. A total of 451 (6 percent) citations proceeded to full-text screening;ten (0.1 percent) were included in the final evidence base. No citations predicted to be irrelevant by Abstrackr were included in the final evidence base.ConclusionsText-mining tools such as Abstrackr have the potential to reduce workload associated with title and screening, without missing relevant citations.